Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve] Improved Docker build #514

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

Conversation

lehcode
Copy link

@lehcode lehcode commented Apr 30, 2024

This commit provides an improved Docker build

  • Adds build caching
  • Adds and configures Miniconda environment with GPU driver support.
  • Adds the ability to sync local and Docker environments with Conda and environment.yml.
  • Adds name to devika-backend-engine service image.
  • Adds hostnames additionally to service names to simplify container communication by shorter and more convenient host names.
  • Implements various environment variables for optimization.
  • Patches source files at the image build stage for seamless Docker support
  • Replace UI package manager with Yarn, which does not require additional installation
  • Replaces base UI image with node:18.20.2-bullseye, which provides the node out-of-the-box.
  • Replaces base image of backend service to python:3.11.9-bookworm with Python 3.11 preinstalled.

This commit provides an improved Docker build.
1. Adds and configures Miniconda environment with GPU drivers support.
2. Adds ability to sync local and Docker environments with Conda and environment.yml.
3. Fixes container communication issues
4. Adds name to `devika-backend-engine` service image
5. Adds hostnames additionally to service names to simplify containers communication by shorter and convenient host names.
6. Implements various environment variables for optimization.
7. Patches source files on image build for seamless Docker support
8. Replaces UI package manager with Yarn, which does not require additional installation
9. Replaces base UI image to `node:18.20.2-bullseye` which provides Node out-of-the-box.
10 Replaces base image of backend service to `python:3.11.9-bookworm` with Python 3.11 preinstalled.
This commit can be safely discarded.
@ARajgor
Copy link
Collaborator

ARajgor commented Apr 30, 2024

remove the lock file

@ARajgor
Copy link
Collaborator

ARajgor commented Apr 30, 2024

also how much each the size of the containers?

@lehcode
Copy link
Author

lehcode commented Apr 30, 2024

Thanks for the prompt reply, @ARajgor!

Preparing another commit with hotfixes.
I knew you'd ask )
All containers that use CUDA are large because CUDA is large.
I coded from the view that size matters at the release stage, while the current one seems to be an active development, so I did not add any cleanup code and added caching, which speeds up rebuilds, even if I suspect that the price is image size or other disk resources.
I know how to optimize images for production later.

@StuckInLoop
Copy link

StuckInLoop commented Apr 30, 2024

devika-backend-engine-1 | Traceback (most recent call last):
devika-backend-engine-1 | File "", line 198, in _run_module_as_main
devika-backend-engine-1 | File "", line 88, in _run_code
devika-backend-engine-1 | File "/home/nonroot/devika/devika.py", line 8, in
devika-backend-engine-1 | init_devika()
devika-backend-engine-1 | File "/home/nonroot/devika/src/init.py", line 7, in init_devika
devika-backend-engine-1 | logger = Logger()
devika-backend-engine-1 | ^^^^^^^^
devika-backend-engine-1 | File "/home/nonroot/devika/src/logger.py", line 11, in init
devika-backend-engine-1 | config = Config()
devika-backend-engine-1 | ^^^^^^^^
devika-backend-engine-1 | File "/home/nonroot/devika/src/config.py", line 11, in new
devika-backend-engine-1 | cls._instance._load_config()
devika-backend-engine-1 | File "/home/nonroot/devika/src/config.py", line 23, in _load_config
devika-backend-engine-1 | with open("sample.config.toml", "r") as f:
devika-backend-engine-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
devika-backend-engine-1 | FileNotFoundError: [Errno 2] No such file or directory: 'sample.config.toml'
devika-backend-engine-1 exited with code 1

@lehcode
Copy link
Author

lehcode commented Apr 30, 2024

I'm not sure if this is related to this specific PR.
Original sample.config.toml must exist. Check if working directory contains it in container

@lehcode
Copy link
Author

lehcode commented May 1, 2024

  • Removes bun lock file
  • Fixes missing playwright browser error on task completion

@lehcode
Copy link
Author

lehcode commented May 1, 2024

@ARajgor docker-compose build should run smoothly now

@ARajgor
Copy link
Collaborator

ARajgor commented May 1, 2024

you didn't remove the yarn.lock

@lehcode
Copy link
Author

lehcode commented May 1, 2024

@nalaso
Copy link
Contributor

nalaso commented May 1, 2024

devika is using bun as it default package manager. bun.lockb is already included in the repo.

@lehcode
Copy link
Author

lehcode commented May 1, 2024

@nalaso, the reasoning is mentioned in the PR description:
"Replace UI package manager with Yarn, which does not require additional installation"
I don't care about specific PM. Feel free to revert bun.lock and npm install bun back.

@Polygons1
Copy link

how about using oven/bun? also the app can work with just Bun

@lehcode
Copy link
Author

lehcode commented May 2, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants